A Music Stream Segregation System Based on Adaptive Multi-Agents

A principal problem of auditory scene analysis is stream segregation: decomposing an input acoustic signal into signals of individual sound sources included in the input. While existing signal processing algorithms cannot properly solve this inverse problem, a multi-agent-based architecture has been considered to be a promising methodology in its modularity and scalability. However, most attempts made so far depend on subjectively defined rules to deal with variability of sounds. Here we propose a quantitatively principled architecture in agent interaction by formulating the problem as least-squares optimization. In this architecture , adaptation of the agents is the essential idea. We have developed two kinds of processing to realize adaptivity: template filtering and phase tracking. These mechanisms enable each agent to optimally, in the least-squares sense, track the individual sound. As an example application of the proposed architecture, we have built a music recognition system that recognizes instrument names and pitches of the notes included in ensemble music performances. Experimental results show that these adaptive mechanisms significantly improve the recognition accuracy. 1 Introduction In recent years scene analysis based on acoustic information , termed auditory scene analysis, has received a renewal of interest. Recognizing external events based on acoustic information is an essential function for systems that work in the real world. A principal problem toward auditory scene analysis is stream segregation [Bregman, 1990]. This segregation means decomposing an input signal into signals of individual sound sources included in the input. However, once multiple acoustic signals are mixed up, their segregation is, so far, considered very difficult because it is an ill-posed inverse problem. Nevertheless, technical and applicational importance has attracted researchers to this field of study. Specifically , works intended to model integration of bottom-up and top-down processing includes [Lesser et a/. These works are characterized by their architectures based on processing modules with simplified functions and communications between these modules, which we call a multi-agent architecture. While the architecture intrinsically enjoys modularity and scalability, quantitative background for behavior of agents is not yet established. Practically, the multi-agent based systems mentioned above require subjectively defined rules to control interaction schemes or to adjust parameters for modules in order to deal with variations of sounds. Here we propose a quantitatively principled architecture , called Ipanema, designed to solve the stream segregation problem for sound mixtures. The essential idea is adaptation of agents to cope with variation of a sound. We have developed …